Converting monoscopic, or 2D, image sequences to stereoscopic, or 3D, sequences may be accomplished through any one of the numerous existing processes. One of the most common conversion processes requires that discrete elements of an image be segmented so that the discrete elements can then be individually modified during the conversion process. During the conversion process, these individual discrete elements are offset horizontally and re-composited into a new position. The new position is determined by referencing the inherent monoscopic depth cues within the image sequence itself, by artistic design, or a combination of both methodologies. The segmentation, modification, and compositing steps create an alternate viewing perspective of the original image sequence.
The creation of an alternate view may be performed once, in which case the observer views the original 2D image with one eye, and the second modified alternate view with the other eye. Or, the creation of an alternate view can be carried out twice, with the horizontal modifications and offsets being carried out in opposite directions between the two alternate views. In this latter case, two entirely new viewing perspectives are created, and the observer views one of them with one eye and the other with the second eye.
The above illustrated process of generating the new and alternative view(s) from a single original monoscopic image sequence reveals newly viewable areas within the alternate view(s), which were formerly occluded by the discrete elements. Left as is, these formerly occluded areas are distracting to the viewer and need to be replaced with image data that is appropriate for that image. What replaces these formerly occluded portions of the new alternate image varies depending on the process and/or what is logically behind the discrete element that was blocking the formerly occluded portion. Examples of what may replace the formerly occluded area include a solid color, replication of context image data, or an output of a selected algorithm.
Current 2D to 3D conversion processes have to deal with the problem of previously occluded and newly viewable regions. As such, prior conversion processes have not only attempted to establish an effective conversion process, but also methods of treating these revealed, formerly occluded surfaces after the 3D (i.e., “alternate view”) images are created. Several of these methods are discussed herein below.
One such method of dealing with revealed surfaces is automatic “inpainting” the formerly occluded area to replace the missing data. However, inpainting is computationally expensive and typically the results are inaccurate and requires additional input from a sentient operator. Another method is manually adjusting image data after the conversion process. Unfortunately, manually adjusting image data after the conversion process can be equally time consuming and expensive, as it requires repetitious practice to prevent inaccuracies. In both cases, replacement image data must match the image data within the newly created image sequence and also complement the alternate perspective view, whether the alternate view was also generated in the conversion process or is the original monoscopic image sequence.
Thus, there remains a long felt need in the art for methods and systems of efficiently and effectively treating occlusions and transparencies resulting from 2D to 3D conversion processes.